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A SYSTEM FOR CREATING MICROSCOPIC DIGITAL MONTAGE IMAGES 
CROSS-REFERENCE TO RELATED APPLICATIONS 

The present application is a continuation-in-part of U.S. Patent Application No. 
09/757,703, filed January 11, 2001, U.S. Patent Application No. 09/758,037, filed 
January 1 1, 2001, and U.S. Patent Application No. 09/788,666, filed February 21, 2001, 
all of which are currently pending and assigned to the assignee of the present invention. 
FIELD OF THE INVENTION 

The present invention relates to microscopic digital imaging of complete tissue 
sections for medical and research use. In particular, it describes a method for high 
throughput montage imaging of microscope slides using a standard microscope, digital 
video cameras, and an illumination system. 
BACKGROUND OF THE INVENTION 

Laboratories in many biomedical specialties, such as anatomic pathology, 
hematology, and microbiology, examine tissue under a microscope for the presence and 
the nature of disease. In recent years, these laboratories have shown a growing interest in 
microscopic digital imaging as an adjunct to direct visual examination. Digital imaging 
has a number of advantages including the ability to document disease, share findings, 
collaborate (as in telemedicine), and analyze morphologic findings by computer. Though 
numerous studies have shown that digital image quality is acceptable for most clinical 
and research use, some aspects of microscopic digital imaging are limited in application. 

Perhaps the most important limitation to microscopic digital imaging is a "sub- 
sampling" problem encountered in all single frame images. The sub-sampling problem 
has two components: a field of view problem and a resolution-based problem. The field 
of view problem limits an investigator looking at a single frame because what lies outside 
the view of an image on a slide cannot be determined. The resolution-based problem 
occurs when the investigator looking at an image is limited to viewing a single resolution 
of the image. The investigator cannot "zoom in" for a closer examination or "zoom out" 
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for a bird f s eye view when only a single resolution is available. Significantly, the field of 
view and resolution-based problems are inversely related. Thus, as one increases 
magnification to improve resolution, one decreases the field of view. For example, as a 
general rule, increasing magnification by a factor of two decreases the field of view by a 
5 factor of four. 

To get around the limitations of single frame imaging, developers have looked 
at two general options. The first option takes the form of "dynamic-robotic" imaging, in 
which a video camera on the microscope transmits close to real time images to the 
investigator looking at a monitor, while the investigator operates the microscope by 
10 remote control. These systems have been used successfully in initial telepathology 

collaborations by allowing a distant consultant to view the specimen without the delays 
uJ and losses associated with sending the physical slide to the consultant for review, and by 

S I 

[fj allowing the consultant to view the entire slide, not just a few static images captured by 

!j! the initial user. 

III 15 However, these systems do not lend themselves to significant collaborations, 

l«j documentation or computer based analysis. To be successful, remote transmission 

y\ requires lossy video compression techniques to be used in order to meet the network 

jU bandwidth requirements, or requires significant delays in the image display if lossless 

T: transmission is used. In addition, lossy compression on the order required for real-time 

20 remote transmission, severely limits computer-based analysis, as well as human 

diagnosis, due to the artifacts associated with lossy compression techniques. Remote 
operation of a microscope also requires only a single user to use the instrument at one 
time, requiring instrument scheduling and local maintenance of the instrument and the 
slides to be viewed. 

25 The second option being investigated to overcome the limitations inherit in 

single frame imaging is a montage (or "virtual slide") approach. In this method, a robotic 
microscope systematically scans the entire slide, taking an image at each "camera field" 
corresponding to the field of view of the camera. Camera field and field of view shall 
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hereinafter be referred to as the "field." The individual images are then "knitted" together 
in a software application to form a very large data set with very appealing properties. 
The robotic microscope can span the entire slide area at a resolution limited only by the 
power of the optical system and camera. Software exists to display this data set at any 
5 resolution on a computer screen, allowing the user to zoom in, zoom out, and pan around 
the data set as if using a physical microscope. The data set can be stored for 
documentation, shared over the Internet, or analyzed by computer programs. 

The "virtual slide" option has some limitations, however. One of the limitations 
is file size. For an average tissue section, the data generated at 0.33 jam/pixel can be 

10 between two and five gigabytes uncompressed. In an extreme case, the data generated 
from one slide can be up to thirty-six gigabytes. 

A much more difficult limitation with the prior systems is an image capture time 
problem. Given an optical primary magnification of twenty and a two-third inch coupled 
device or "CCD", the system field of view is approximately (8.8 mm x 6.6 mm) / 20 = 

15 0.44x0.33 mm. A standard tissue section of approximately 2.25 square centimeters, 

therefore, requires approximately fifteen hundred fields to capture an image of the entire 
tissue section. 

Field rate, which is the amount of time it takes to capture an image of a field 
and set-up the apparatus capturing the field for a following image capture, in montage 

20 systems is limited by three factors - camera frame rate (the number of camera images 
acquired per second), image processing speed (including any time required to read the 
camera data, perform any processing on the camera data prior to storage, and to store the 
final data), and rate of slide motion, which is the time required for the slide to be 
mechanically repositioned for the next image acquisition. Given today's technology, the 

25 rate of slide motion is a significant limiting factor largely because the existing imaging 

systems require the slide to come to a stop at the center of each field to capture a blur free 
image of the field. 
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For example, traditional bright field microscopic illumination systems were 
designed to support direct visual examination of a specimen on the field and therefore 
depend on a continuous light source for illumination. Continuous light, however, is a 
significant limitation for digital imaging in that the slide, which must move to capture an 
entire image, but must be stationary with respect to the camera during CCD integration, 
thus moving the slide from the light. Moreover, slide motion during integration results in 
a blurred image. Traditional montage systems, therefore, have had to move the slide (and 
stage) from field to field in a precise "move, stop, take image and move again" pattern. 
This pattern requires precise, expensive mechanics, and its speed is inherently limited by 
the inertia of the stage. 

The three-dimensional characteristic of a typical tissue sample and the slide 
places additional limitations on the imaging system. Like all lenses, microscope optics 
have a finite depth of field - the distance within which objects will appear to be focused. 
A typical depth of field is about 8 microns for a 10x objective, and in general, as the 
magnification increases, the depth of field decreases. While microscope slides are 
polished glass, the flatness of the slide can vary on the order of 50 microns or more 
across the slide. The variations in the tissue sample thickness and any defects associated 
with placing the sample on the slide, such as folds in the tissue, also affect the optimal 
position of the slide with respect to the imaging optics. The magnitude of the optimal 
position and the limited depth of field of the microscope optics require the focus to be 
adjusted as the system moves from field to field. The time to refocus the system at each 
field also contributes to the overall capture time of the montage image. 

Thus, a system is needed to reduce the image capture time. The system must 
also enable efficient and high quality imaging of a microscope slide via a high-resolution 
slide scanning process. 
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SUMMARY OF THE INVENTION 
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The present invention relates to a method and system for creating a high 
throughput montage image of microscope slides. The system includes an optical system, 
components that are used in a pre-scan processing, and components for auto-focusing by 
enabling accurate focus control of optical elements without requiring the stage to be 
stopped and refocused at each tile location. The optical system includes at least one 
camera, a motorized stage for moving a slide while an image of the slide is captured, a 
pulsed light illumination system that optically stops motion on the motorized stage while 
allowing continuous physical movement of the motorized stage, and a stage position 
detector that controls firing of the pulsed light illumination system at predetermined 
positions of the motorized stage. The components that are used in the pre-scan 
processing include an image-cropping component, a tissue finding component and a scan 
control component. The image-cropping component and tissue finding component 
identify tissue regions on the slide in the optical system and determine exact locations of 
tissue on the slide. The scan control component uses information about the locations to 
generate control parameters for the motorized stage and the camera. The components for 
auto-focusing include a focal point selection component, a focal surface determination 
component, and a scan component. The focal point selection component and the focal 
surface determination component use the control parameters to ensure that a high-quality 
montage image is captured. The scan component is able to capture a high-quality 
montage image by maintaining motion of the motorized stage and synchronization of the 
optical system. The scan component controls the stage position to maintain in-focus 
imaging during the scanning process without stopping the stage and refocusing at each 
location and fires a pulsed-illumination source at the appropriate position to guarantee 
image alignment between sequential camera images. 

Accordingly, it is a benefit of the invention that it provides a microscopic 
imaging system for whole slide montage in which standard microscope optics, off the 
shelf cameras, a simple motorized stage, and a pulse light illumination system can be 
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used to produce precisely aligned image tiles, and acquire these image tiles at a speed 
limited primarily by the camera frame rate. 

The present invention uses a strobe light, triggered by a direct Ronchi ruler or 
other stage-positioning device, to produce precisely aligned image tiles that can be made 
5 into a montage image of tissue sections on a microscope slide. Significantly, due to the 
short light pulse emitted by a strobe, clear images can be obtained without stopping the 
microscope stage. This significantly increases the image throughput while decreasing the 
expense and precision required in the stage mechanics. 

In one embodiment, a strobe arc is placed at the position of the lamp bulb in a 
10 standard microscope system. The camera shutter is opened and the strobe is fired in 

response to the position of the stage as reported by a direct position sensor. If stray light 
%J is minimized, the camera exposure can be much longer than the strobe flash, allowing 

hfj low cost cameras to be utilized. 

"j* It is another benefit of the invention to significantly increase the image throughput 

III 15 of a tiling image system by allowing, through the use of the strobe light, continuous 

r j motion of the slide under the microscope. The inventive system thus eliminates the need 

a "-* to stop the microscope stage to capture an image. 

U-J 

H It is another benefit of the invention to reduce the demands of camera, stage, and 

PI 

2 strobe synchronization by controlling the firing of the strobe light based on direct stage 

20 position feedback, thereby substantially reducing the mechanical precision required of the 
stage and camera components. 

It is another benefit of the invention to use a pre-scan process applied to a 
macroscopic image of the entire slide, to guide a high-resolution slide scanning process 
and ensure high-quality images of the entire specimen are acquired. The pre-scan process 
25 includes an image cropping component, a tissue finding component, a scan control 

component, a focus point selection component, a focal surface determination component, 
and a scan component. The image cropping and tissue finding components identify 
interesting regions on the slide to be scanned. The focus point selection and focal surface 
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determination components ensure that a high quality image is captured by the scanning 
process, by enabling accurate focus control to be maintained. 

It is another benefit of the invention to use a high-resolution slide scanning 
process to control the operation of the motorized stage, camera and strobe illumination. 
This process utilizes information gathered by the pre-scan process, namely the imaging 
regions and focus parameters, to control the positioning of the stage to image only the 
regions of interest and to ensure the individual images are well aligned and in focus. 

Additional features and advantages of the invention will be apparent from the 
description that follows, or may be learned by practice of the invention. The objectives 
and advantages of the invention to be realized and attained by the microscopic image 
capture system will be pointed out in the written description and claims hereof as well as 
the appended drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are included to provide a further 
understanding of the invention and are incorporated in and constitute a part of this 
specification, illustrate embodiments of the invention that together with the description 
serve to explain the principles of the invention. 

Fig. 1 is a partial isometric view of an embodiment of an imaging apparatus of the 
present invention; 

Fig. la is a front view of the apparatus of Fig. 1 ; 

Fig. lb is a side view of the apparatus of Fig. 1; 

Fig. lc is a top view of the apparatus of Fig. 1; 

Fig. 2 is a macroscopic image resulting from operation of an embodiment of the 
cropping component; 

Fig. 3 illustrates a result of the tissue finding component; 

Fig. 4 is an overlay of Figs. 2 and 3 illustrating the regions of the slide to be 
imaged; 
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Fig. 5 illustrates a result of the focus point selection component on a sample 

image; 

Fig. 6 is a generated three-dimensional data set for the image of Fig. 5; and 

Fig. 7 is a set of charts graphically depicting steps implemented in an embodiment 
of the inventive system. 

DESCRIPTION OF THE PREFERRED EMOBODIMENTS 

Reference will now be made in detail to the preferred embodiments of the present 
invention, examples of which are illustrated in the accompanying drawings. The following 
paragraphs describe the functionality of the inventive system and method for focus 
controlled, high throughput montage imaging of microscope slides using a standard 
microscope, camera, and a motorized mechanical stage. 

Fig. 1 illustrates a first embodiment of the imaging apparatus of the present 
invention. Figs, la-lc illustrate front, side, and top views, respectively, of the imaging 
apparatus illustrated in Fig. 1 . It should be noted that not all components that may be 
included in an imaging apparatus are illustrated in Figs. 1-lc. For example, a stage 
support that attaches the stage to the imaging apparatus is not illustrated so as not to 
obstruct the view of other components. In this embodiment, a slide 1 12 to be imaged is 
placed on a thumbnail imaging position in a slide holder on a motorized stage 102. A 
single frame image containing the entire slide may be taken with a macro camera 106. 
One skilled in the art will recognize, however, that a suitable image may alternately be 
captured by combining multiple images taken by a microscopic camera such as 
microscopic camera 104. Microscopic and macroscopic images may furthermore be 
captured by use of two separate cameras or through the use of a single camera. The low- 
resolution image may be analyzed by software components, described below, to 
determine the locations of tissue on slide 112. This information can then be used to 
generate control parameters for stage 102 and microscopic camera 104 to ensure that the 
scanning process captures high quality images of only the tissue regions, substantially 
reducing the time to scan an average slide. 
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One skilled in the art will recognize that, although capturing the single 
macroscopic image saves time, it is not necessary for the operation of the invention. 
However, where a macroscopic image is to be used, multiple macroscopic images may be 
required to generate control parameters to the accuracy required based on the ratio of the 
macroscopic to microscopic magnifications and the camera specifications of each 
camera, if separate cameras are utilized. 

Once the regions to be microscopically imaged are identified, the slide is scanned 
under the microscope optics. To facilitate rapid imaging of the slide and to avoid the stop 
image reposition delays associated with traditional imaging systems, a high-speed strobe 
light is used to optically stop the motion of the stage, thus the slide specimen, while 
allowing continuous stage motion. It should be apparent to one skilled in the art, that any 
pulsed illumination system may be used in place of the high-speed strobe light. To 
eliminate overlap or missed tissue between the microscope images, precise alignment of 
the stage and camera, along with accurate stage positioning, and camera and strobe 
synchronization, are required. To reduce the camera precision requirements, a direct 
stage position sensor is used to control the firing of the strobe, and thus the camera 
exposure. 

In this fashion, the camera can be operated with a long exposure window in 
comparison to the very short strobe flash, allowing lower cost components, specifically 
the stage and camera, to be utilized. 

Specifically in an embodiment illustrated in Figs. 1-lc, a pre-scan processing of 
the low-resolution or thumbnail image includes an image cropping component, a tissue 
finding component, a scan control component, a focus point selection component, and a 
focal surface determination component. Those pre-scanning components may then be 
followed by a scan component. The image cropping component and tissue finding 
component identify tissue regions on the slide to be scanned. The scan control 
component generates the necessary control parameters to scan only the regions of interest 
under the microscope optics. The focus point selection component and a focal surface 
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determination component ensure that the scan component captures a high-quality 
montage image, by enabling accurate focus control of the optical elements without 
requiring the stage to be stopped and refocused at each tile location, substantially 
reducing the acquisition time. 

One step in processing the thumbnail image consists of flat-field correcting the 
macroscopic thumbnail image using a similar image that may have been obtained from 
the same camera and a blank slide. This removes any spatial light anomalies from the 
thumbnail image, which may reduce the efficiency of the tissue finding component. 
Depending upon the format, or size, of the camera and the aspect ratio of the slide, a 
portion of the image may contain non-slide objects such as a slide carrier (not shown in 
the figures). To remove these features, the thumbnail image may be cropped to remove 
non-slide objects, thus retaining only the slide information. 

The image cropping may be accomplished via a two-pass process. The first pass 
in such a process determines an approximate location of the slide boundary, and the 
second pass fine-tunes this estimate. The search for the boundary is conducted over 
upper and lower intervals corresponding to the regions expected to contain the upper and 
lower slide edges, respectively. The slide or region of interest may be assumed to be 
positioned near the center, vertically, in the thumbnail image. The portion of the image 
falling outside of the identified slide boundary is removed. It should be noted that the 
cropping component and each of the other components described herein may operate on 
either a grayscale or color image. The image may also be cropped at the estimated edge 
locations, and then uniformly reduced in size to produce a small thumbnail image of the 
slide for rapid, visual slide identification. 

Because the slide may not be oriented perfectly horizontally in the original 
thumbnail image, the identified slide edges are likely to lie at an angle. Thus, even after 
cropping, there may be remnants of the slide edges or cover slip in the cropped image. 
Therefore, the image cropping component attempts to identify pixel blocks that likely 
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contain these remaining edges and flags these blocks as edges that will not be considered 
for high resolution imaging by the tissue finding component. 

The resulting cropped image generated by the image cropping component may 
serve as an input to the tissue finding component. This component locates regions in the 
5 thumbnail image that contain tissue of interest to a specialist. In order to minimize the 
time and storage space required to accomplish high-resolution slide imaging, the 
inventive system may capture only those regions of the slide that contain tissue. This 
approach may be facilitated by identifying regions containing tissue in the thumbnail 
image. 

10 The tissue finding component identifies tissue regions via a sequence of filters 

that incorporate knowledge of the typical appearance and location of tissue and non- 
hi tissue slide regions. Initial filtering steps, in one embodiment, convert the image to a 
\ti grayscale image and analyze the mean and standard deviation of the local pixel 
ell intensities. Pixel mean intensities may be used to differentiate tissue-containing regions 
s U 1 5 from blank and other non-tissue regions, such as those containing the slide label or other 
#*j markings. The standard deviation data may represent the amount of variation in pixel 

l*\ intensity values and thus is a good indicator of the border between tissue and the blank 

%y 

H slide. The mean and standard deviation data is combined to generate a threshold value 

that is used to make an initial classification of tissue versus non-tissue. Subsequently, 

20 morphological filters may be applied to refine the classification based on the size and 
position of neighboring groups of potential tissue pixels. 

The embodiment described above uses the mean and standard deviation of the 
local pixels as the basis for detecting regions of interest. One skilled in the art will 
recognize, however, that other image characteristics can also be used to identify the 

25 specimen from non-items of interest such as dust and scratches. That embodiment may 
also process a grayscale macroscopic image. It should be noted, however, that the pixel 
intensity differentiation tools described herein can be applied to each of the color 
components (traditionally, red, green and blue) of a color image in addition to being 
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applied to a grayscale image. Additional processing tools can also be applied between 
color components to refine the tissue finding accuracy and to remove features such as 
labels and writing that are not critical to the application, or to select user defined areas of 
interest to be scanned, such as regions circled by a purple marker. 

The filters, which comprise the tissue finding component, process the pixels of the 
cropped grayscale thumbnail image in groups that correspond to slide regions, or tiles, 
that can be imaged individually during the high-resolution scanning process. These 
filters ensure that tiles only partially filled with tissue are classified as tissue-containing 
tiles. The final output of the filter sequence is a tiling matrix, the value of which 
indicates which tiles should be imaged. The tiling matrix subsequently guides the high- 
resolution scanning process. 

An example of the image cropping and tissue finding processes are shown in 
Figs. 2, 3 and 4. Fig. 2 illustrates the macroscopic image after flat-field correction and 
image cropping. Fig. 3 illustrates the results of the tissue finding component. The 
resulting tile matrix shown in Fig. 3 has a one-to-one correspondence to the field of view 
of the microscopic camera. White pixels (binary 1) signify fields to be captured and 
imaged, and black pixels represent regions not to image. Fig. 4 illustrates an overlay of 
Figs. 2 and 3 representing the sections of the slide to be imaged. For the application 
depicted in Figs. 2-4 (anatomical pathology), it is imperative to image all suspect regions 
that may contain tissue so conservative criteria are used in the tissue finding component, 
resulting in cover slip edges and writing etched into the slide to be identified as to be 
imaged. The savings in the acquisition time is represented by the ratio of the white to 
black areas of Fig. 3. For this image, only 53% of the slide region is to be imaged, 
including the label and cover slip edges, and etched writing on the slide. 

At the completion of the tissue finding component, the scan control component 
interprets the tissue finding tile matrix (Fig. 3) and transposes the positions into actual 
stage coordinates for the microscopic imaging. A program running on a host computer 
controls the operation by communicating with a stage controller and microscopic camera 
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104. Actual scanning can occur in any fashion such as by rows or columns, or in a step 
fashion to image neighboring areas. 

To achieve good focus for an entire slide, the surface that best represents the 
focal position of the sample with respect to the optical elements may be determined and 
used to automatically control the focus as the tissue is scanned under the microscope 
optics. These steps may furthermore be completed under the focus point selection 
component and the focus surface determination component. 

In one embodiment, the focus point selection component evaluates the tissue 
regions of the thumbnail image and selects several points on which to initially focus the 
microscope optics. In this embodiment, that focus point selection is based on pixel 
intensity and distribution. Initially, tissue containing pixels or groups of tissue containing 
pixels are considered. Those pixels or pixel groups will be referred to hereinafter as 
regions. The regions that will be focussed upon, or focus regions, are then selected based 
on the relative contrast between the intensity of those regions within the thumbnail image 
and the distribution of those regions within the tissue containing portions of the image. It 
will be recognized that other intensity and/or distribution selection criterion may be 
utilized to select focus regions. After the focus regions are identified, a normalized focus 
surface such as a plane or curve may be drawn through the focus points and an equation, 
model, curve, or other method for describing appropriate focal lengths along that surface 
can be obtained. Proper focus for each region to be captured by the high-resolution scan 
may then be calculated by focusing on, for example, the plane or curved surface. 

Selecting the appropriate focus region based on relative contrast improves the 
likelihood that regions that contain tissue will be selected over a method of picking focus 
regions based on a pre-selected grid-pattern. Selecting regions based on their overall 
distribution with respect to the tissue coverage area provides assurance that the resulting 
plane or surface will be representative of the entire slide and not limited to a small 
portion of the tissue, as could occur if selection were based solely on the relative contrast 
of pixels. 
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To select focus regions based on contrast, the present invention may select a 
number of focus regions having a desired contrast quality. For example, six to ten of the 
darkest regions may be selected from a back-lit image under the assumption that dark 
regions contain a large amount of tissue. 

Distribution refers to the overall distribution of regions with respect to the tissue 
coverage area. Because a surface defined by the focus regions is the basis for 
maintaining the focus across the entire specimen during scanning, it is beneficial to have 
focus regions dispersed across the specimen rather than being grouped in close proximity 
to one another in one or more areas of the specimen. In that way, every point to be 
scanned will be close to a focus point. Thus, the scanned optical position at each point, 
as defined by the surface, should be very nearly the optimum in-focus position for that 
point. 

The focus point selection component may, for example, assure that at least one 
focus point is located within each of a number of pre-selected areas of the tissue. Those 
areas may, for example, be separate pieces of tissue on a slide. If a focus point does not 
exist on each pre-selected area, the focus point selection component may select additional 
regions, for example, having the desired contrast quality, until at least one focus point is 
identified on each pre-selected area. The number of data points required will depend on 
the actual three-dimensional structure defined by the specimen and slide, and the 
geometrical dimension of the surface to be fit. Once the surface has been determined, an 
error function can be calculated to determine the fit accuracy. For example, the mean 
square error of each selected focal region may be calculated to determine how much error 
exists between the surface fit and each focus region. If that error is greater than a 
predetermined acceptable error level, at one or more points, additional data points may be 
added to the calculation and/or points that have large errors may be eliminated and the 
surface may be recalculated under the assumption that the points having excessive errors 
were anomalies. A surface may then be fitted to those points to define a focal surface to 
be utilized when scanning the tissue. 
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In alternative embodiments, the focus points are either user definable through an 
input file or through a suitable user interface. In addition, for cases where the specimen 
locations are reproducible on the slides, the focus points can be predefined and repeated 
for each slide without the use of a macroscopic image or any preprocessing to find the 
5 tissue regions. 

Once selected, each focus position is placed under the microscope optics in the 
focal surface determination component, and an auto-focus program determines the best- 
fit focus at each position. This generates a three-dimensional data set corresponding to 
the optimal specimen distance at each stage location. These data points are used as input 
10 to a surface fitting routine that generates the necessary control parameters for the slide 
scanning process. 



m 
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P At the completion of the focus point selection, the tissue information and the 

surface parameters are passed to the scan control component. This component is 
responsible for the motion of the stage and synchronization of the microscopic imaging 
^ 15 system during montage image acquisition. To achieve accurate, well-aligned tiled 
r j images, the specimen must be positioned such that each camera image is aligned within 

the equivalent single pixel spacing in real or stage space (camera pixel size divided by the 
overall optical magnification). This usually entails a stage step of bx and by where each 
step is directly related to the effective pixel size, which may be expressed in terms of 
20 camera pixel size / optical magnification, and the number of image pixels in the x and y 
directions respectively. For example, a 1300x1030 pixel, 10 jam square pixel camera 
operated at 20x magnification results in bx = 10 \im * 1300 / 20 = 650 jam and 5y = 10 
Hm * 1030 / 20 = 515 |im. To maintain focus during the scanning process, the stage must 
be positioned at the proper focal position as determined by the focus surface parameters: 
25 zjj = /(xj , yj), where Zy is the vertical position of the slide with respect to the optical 

components, f(x[ , yj) is the function used to represent the best focus surface, and xj and 
yj are the positions of each camera image in the x and y axes, respectively. The camera 
image positions can be expressed as linear relations between the starting position of the 
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stage (x G and y Q ) and the step size associated with each image dimension: 
xi = x Q + i * 6x and yj = y Q + i * 5y. 

Image montage scanning is traditionally accomplished by either scanning by rows 
or columns. Assuming that tiling is completed by scanning across a row in the x- 
5 direction and stepping vertically in the y-direction after each row has been scanned, the 
stage is simply positioned at the appropriate position given by z[ j = f(x[ , yj). Thereafter, 
the stage is stopped and an image is acquired. If imaging is accomplished during 
continuous motion of the stage in the x-direction via a line scan camera or alternative 
imaging arrangement, the vertical velocity as a function of focal position xj and time, can 

Q 10 be computed from the partial derivative of the focal surface: V z (v x , yj) = 9f(xi, yj)/9x * 

u3 v x , where V z is the velocity of the vertical position of the stage and v x is the velocity in 

H 

k % ?\ the x-direction. The velocity of the vertical position of the stage can be used to control 

?* 

"1% the optical position and maintain focus as images are acquired continuously across the 

n\ 

3 " row. 

■2 

rj 15 Fig 5. represents the results of the focus point selection component. This figure 

5 '*j shows the thumbnail or macroscopic image of the region to be scanned. The light spots 

a s I 

504 overlaid on the specimen 506 represent the positions selected by the focus point 
selection component. These positions are placed under the microscope and auto-focused 
on each location. Fig. 6 illustrates the three-dimensional data set generated by focussing 
20 on each of the focus points of the specimen depicted in Fig. 5. For this slide, the best fit 
was planar, z(x,y) = dz/dx x + dz/dy y + z0, where dz/dx (dz/dy) is the slope or pitch of 
the plane with respect to x-axis (y-axis) and zO is the vertical offset of the plane with 
respect to the z-axis. The best fit parameters for the specimen of Fig. 5 are also shown in 
Fig. 6. 

25 At the completion of the pre-scan processing, the tile matrix and the stage control 

parameters are passed to a scanning-process control program. The scanning process 
control program controls the operation by communicating with a stage controller, a stage 
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position sensor, a camera and the strobe firing circuitry. In the invention, the computer 
program controls the operation of stage 102, camera 104 and strobe 108 illumination. 
The actual slide scanning can be automated to image entire slides, image only a portion 
of the slide or use a user-interface to allow the user to select the regions to be imaged. 
5 Once a region has been selected for imaging, the program then controls the operation by 
communicating with a stage controller, a stage position sensor, camera 104 and strobe 
firing circuitry 108. Preferably, tiling is performed by moving stepwise along the short 
axis and with continuous motion along the long axis. In other words, tiling is done one 
row at a time. For this reason, a stage position is monitored and controlled differently 
10 along each stage axis. Along the short axis of the slide, the stage position is monitored 
and controlled, by the program, directly through the stage controller. Along the long 
axis, however, the stage position is monitored by a direct stage position sensor, which can 
be separate or part of the overall stage control circuitry. 

In another embodiment, a Ronchi ruler attached to stage 102 is used for the 
1 5 stage position sensor. Those skilled in the art will recognize that any position sensor may 
be used in the invention. This sensor can be external to the stage controller or the 
positional information can be acquired directly from the stage controller with or without 
feedback. 

For reference, a Ronchi ruler is a pattern of alternating light and dark bands, 
20 equally spaced along a substrate, typically either glass or plastic. A position sensor based 
on the Ronchi ruler utilizes a light sensor that is mechanically isolated from the ruler. As 
the ruler passes under the sensor, a series of electronic pulses is generated corresponding 
to the alternating light and dark bands of the ruler. These pulses can be used to monitor 
the position and direction of stage 102. 
25 Based on the magnification of the optics and the camera utilized, strobe 108 is 

fired whenever the position sensor determines stage 102 has moved into the field of view 
of the next tile to be captured by the camera 104. The system continues to capture image 
tiles with precise alignment, until the images of all desired files have been captured or the 
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controlling program tells the system to stop. At the end of the capture process, the slide is 
removed and another slide can be inserted. With current technology, the rate-limiting 
step for image capture utilizing the present invention is the data transfer period in the 
camera. 

Fig. 7 illustrates the signals of camera 104, stage 102, optical position detector, 
and strobe 108. Note that in Fig. 7, graphs 202 and 204 the signals from the optical 
position detector represent motion of stage 102, so their timing will vary depending on 
the speed of the stage movement. Where the system is triggered by the location of stage 
102 as reported by the optical position sensor, precise movement of the stage movement 
is not necessary, allowing for the use of low cost stages 102. 

The system can be run in a variety of modes, depending on how the camera is 
controlled. In one embodiment, the stage location, as sensed by a position sensor, fires 
both the camera 104 and the strobe 108 as indicated by the two traces at 204. In an 
alternate embodiment, camera 104 is free running and only strobe 108 is fired by stage 
position as indicated by a single trace at 204. This mode does not depend on uniform 
motion of stage 102 over the area imaged, because the strobe pulse is much shorter than 
the integration time of the camera 104, wherein the integration time is a time during 
which the camera is receiving an image. As long as the correct stage position is reached 
anytime within the integration time of camera 104, an excellent, well aligned image 
results. 

Firing strobe 108 based on direct position information differs from the more 
traditional application of strobe photography. In traditional strobe photography, a strobe 
and camera are synchronized in time, and positional information regarding the objects 
being imaged can be inferred from the relative position within the image. When the 
present invention is operated in a mode wherein the position feedback controls both 
camera 104 and strobe 108, and camera 104 is not free running, each camera frame 
corresponds to an equally spaced positional change, independent of the stage velocity 
(speed and time variations in the speed). In the case that camera 104 is free running, the 
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stage speed has to be matched to the camera frame rate only to the accuracy such that the 
strobe pulse does not fall outside the exposure window. The relative time within the 
exposure window is irrelevant. 

As will be understood by one skilled in the art, while the present invention 
describes a microscopic optical arrangement, the invention can also be applied to other 
optical imaging, inspection and illumination systems that are used for building up an 
image by matching the stage speed with the camera speed. 

The foregoing description has been directed to specific embodiments of this 
invention. It will be apparent, however, that other variations and modifications may be 
made to the described embodiments, with the attainment of some or all of then- 
advantages. Therefore, it is the object of the appended claims to cover all such variations 
and modifications as come within the true spirit and scope of the invention. 
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